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SIMPLE  MODELS  IN  STOCHASTIC  PRODUCTION  PLANNING 

by 

Suresh  P.  Sethi  and  Gerald  L.  Thompson 

ABSTRACT 

A simple  stochastic  production-inventory  model  with  quadratic  cost 
functions  is  analyzed  in  detail.  The  inventory  process  is  assumed  to  be 
driven  by  a white  noise  process  resulting  into  an  Ito  stochastic  differential 
equation.  Both  finite  and  infinite  horizon  versions  of  the  problem  are 
treated  by  a methodology  based  on  the  theory  of  stochastic  integrals  and 
differentials.  Particular  attention  is  given  to  illustrate  the  methodology, 
which  is  quite  general  and  capable  of  dealing  with  more  complicated  problems. 
The  paper  concludes  with  some  remarks  in  connection  with  the  relationship 
of  the  results  of  this  paper  to  the  results  in  the  deterministic  case. 
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SIMPLE  MODELS  IN  STOCHASTIC  PRODUCTION  PLANNING 


by 

Suresh  P.  SeChi  and  Gerald  L.  Thompson 

1.  INTRODUCTION 

In  an  earlier  paper  [4] , we  considered  a production-inventory 
model  which  determines  production  levels  over  time  to  minimize  a discounted 
quadratic  loss  function.  The  loss  function  was  defined  in  terms  of  the 
deviations  of  production  and  inventory  levels  from  their  rated  or  factory- 
optimal  values.  In  this  paper,  we  discuss  simple  stochastic  extensions 
of  our  earlier  paper.  We  will  obtain  closed-form  solutions  for  both  finite 
and  infinite  horizon  versions  of  the  stochastic  production-inventory  model. 
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B ■ salvage  value  per  unit  terminal  Inventory 
r(t)  ■ the  standard  Wiener  process;  see  [1] 
cr  ■ the  constant  diffusion  coefficient;  see  [l]. 

We  now  state  the  conditions  of  the  model.  The  first  condition  is  the  stock- 
flow  equation  stated  as  an  Ito  stochastic  differential  equation  (see  [1,2,3]): 

dx  ■ (u-S)dt  + adz,  x(0)  ■ xq  . (1) 

We  note  that  process  dz(t)  can  be  formally  expressed  as  §(t)dt,  where  |(t)  is 
considered  to  be  the  white  noise  process  £ 1] . It  can  be  interpreted  as  "sales 
returns,"  "inventory  spoilage,"  etc.  which  are  random  in  nature.  The  second  is  the 
objective  function: 

X 

min  E'{J  e pt[c(u-u.)2  + h(x-x  )2]dt  + e"pT  Bx(T)}  . (2) 

u 0 1 1 

Note  that  we  do  not  restrict  the  production  rate  to  be  nonnegative  as  required 
in  our  earlier  paper  [4].  In  other  words,  we  permit  disposal  (i.e.  u < 0). 

While  this  is  done  for  mathematical  expediency,  we  will  state  conditions  under 
which  a disposal  is  not  required.  Note  further  that  the  inventory  level  is 
allowed  to  be  negative,  i.e.,  we  permit  backlogging  of  demand. 

3.  THE  HAMILTON- JACOBI  EQUATION 

The  solution  of  the  above  model  will  be  carried  out  via  the  development 
of  the  Hamilton- Jacobi  equation  satisfied  by  a certain  'value  function.'  To 
simplify  the  mathematics,  we  assume  that 

Xj  ■ Uj  ■ 0 and  h ■ c ■ 1.  (3) 

This  assumption  results  in  no  loss  of  generality  as  the  following  analysis  can 
be  extended  in  a parallel  manner  for  the  case  without  (3).  With  (3),  we 
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restate  the  stochastic  production  planning  problem: 
T 

max  Etf  - (u2+x2)  e"ptdt  + Be‘pTx(T)], 
0 

subject  to  the  Ito  equation 


dx  * (u-S)dt  + odz,  x(0)  ■ xq.  (5) 

Let  V(t,x)  denote  the  expected  value  of  the  objective  function  from 
time  t to  the  horizon  T with  x(t)  ■ x and  using  the  optimal  policy 
from  t to  T.  The  function  V(t,s)  is  referred  to  as  the  value  function  and 
it  can  formally  be  defined  as 


T 

V(t,x)  ■ max  E[J*  - (u2+x2)e  ptdt  + Be"pTx(T)] 


u t 


given  x(t)  ■ x;  clearly 


V(T,x)  - Bxe”^1. 


By  the  principal  of  optimality,  we  can  write 


V(t,x)  ■ max  E[-(u2+x2)e  ptdt  + V(t+dt,  x+dx)] 


- max  I-(u2+x2)e"ptdt  + E V(t+dt,  x+dx)]. 


Mote  that  we  have  used  dx  and  dt  in  place  of  fix  and  fit  for  convenience. 
We  can  expand  V(t+dt,  x+dx)  using  Taylor's  expansion: 


V(t+dtf  x+dx)  * V(t,x)  + Vfcdt  + Vxdx  + j Vxx(dx)2 


+ higher  order  terms. 


Note  that  we  have  retained  the  term  (dx)  . This  is  an  important  departure 
from  the  deterministic  situations.  Substituting  for  dx  from  (5)  and 
replacing 


(da)‘  by  dt 


derived  in  the  Ito  theory  [1,3],  we  can  write  (9)  as 


V(t+dt,  x+dx)  - V (t,:c)  + (Vt  + Vx(u-S)  + J <T  Vxx)dt 


+ a + higher  order  terms. 


Taking  the  expectation,  we  are  left  with 


EV(t+dt,  x+dx)  - V(t,x)  •+  (Vt  + Vx(u-S)  + | CT  Vxx)dt 


+■  higher  order  terms. 


Since 


E(oVxdz)  - 0. 

Substituting  (12)  in  (8)  and  cancelling  V(t,x)  on  both  sides  yields 
. 0 - max[-(u2+x2)  e"ptdt  + (V£  + Vx(u-S  ) + | ^V^dt 


+ higher  order  terms] 


Dividing  by  dt  and  taking  the  limit  as  dt  - 0,  we  obtain  the  Hamilton- 
Jacobl -Be liman  equation 

0 - max[-(u2+x2)e-pt  +Vf  + Vx<u-S  ) + \ ^V^] . < 
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It  is  now  possible  to  maximize  the  expression  inside  the  bracket  with 
respect  to  u by  taking  its  derivative  with  respect  to  u and  setting  it  to 
zero.  This  procedure  yields 


u 


(15) 


Substituting  (15)  into  (14)  yields  the  following  Hamilton- Jacobi  equation 

ept 

0 - + - x2e"pt  + V - SV  + \ a2V  . (16) 

4 t x 2 xx 

This  is  a partial  differential  equation  which  must  be  satisfied  by  the  value 
function  V(t,x)  with  the  boundary  condition  (7). 

It  is  important  to  remark  that  if  production  rate  were  restricted  to  be 
nonnegative,  then  (15)  would  be  changed  to 

V ept 

u - max  [0,  — ] . (17) 

Substituting  (17)  in  (14)  would  give  us  a partial  differential  equation  which  must  be 
numerically  solved  for  this  case.  We  shall  not  consider  (17)  further  in  this 
paper. 

We  now  turn  to  solving  (16)  in  the  next  section. 

4.  SOLUTION  FOR  THE  UNDISCOUNTED  FINITE  HORIZON  CASE 

Although  (16)  with  the  boundary  condition  (7)  can  be  solved,  it  is 
cumbersome.  To  simplify  matters , we  assume  that 

P - 0 (18) 

for  the  finite  horizon  case.  For  this  case,  we  must  solve  nonlinear  second- 
order  partial  differential  equation 
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0 - ~f-  - x2  + Vfc  - SVx  + j a2  V^,  V (T,x)  = Bx.  (19) 

To  solve  this  equation,  we  let 

V(t,x)  - Q(t)x2  + R(t)x  + M(t).  (20) 

Then 

Vt  - Qx2  + Rx  + M 

Vx  - 2Qx  + R (21) 

V - 2Q 
xx  y 

where  Y denotes  dY/dt.  Substituting  (21)  in  (19)  and  collecting  terms 
gives 

2 

x2[Q  + Q2  - 1]  + x[R  + RQ  - 2SQ]  + M + ^ - RS  + a2Q  - 0.  (22) 

Since  (22)  must  hold  for  any  value  of  x,  we  must  have 


• 2 

Q - 1 - Q , 

Q(T)  - 0 

(23) 

R - 2SQ  - RQ  , 

R(T)  - B 

(24) 

M - RS  - R2/4  - ct2Q, 

M(T)  - 0, 

(25) 

where  the  boundary  conditions  for  the  system  of  simultaneous  differential 
equations  (23),  (24)  and  (25)  are  obtained  by  comparing  (20)  with  the  boundary 
condition  B(T,x)  ■ Bx  of  (19). 

To  solve  (23)  we  expand  Q/(l-Q2)  by  partial  fractions  to  obtain 
• . . 2 1 1-Q  + 1+Q  J ; 


which  can  be  easily  integrated.  The  answer  is 
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where, 


y ■ e 


2(t-T) 


Since  S is  assumed  to  be  constant,  we  can  reduce  (24)  to 


R°  + R°Q  - 0,  R°(T)  - B - 2S 


by  the  change  of  variable  defined  by  R°  * R - 2S.  Clearly  the  solution 
is  given  by 


log  R°  I » - J*  Q(T)dr, 

•t  t 

which  can  be  further  simplified  to  obtain 


R - 2S  + 


2(B-2SJ 


Having  obtained  solution  for  R and  Q,  we  can  easily  express  (25)  as 


M(t)  - - J*  [RS  - R2/4  - a2Q]dt. 
t 


The  optimal  control  is  defined  in  (15),  which  under  the  assumption 
(18)  of  p ■ 0 and  the  use  of  (26)  and  (28)  yields 


- Vx/2  - Qx  + R/2  - S + <y'1?Xy4'+^'2S 


Remarks 

1)  The  optimal  production  rate  in  (30)  equals  the  demand  rate  plus  a cor- 
rection term  which  depends  on  the  level  of  inventory  and  the  distance  from  horizon. 
Since  (y-1)  < 0 for  t < T,  it  is  clear  that  for  the  lower  values  of  x, 
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the  optimal  production  rate  is  likely  to  be  positive.  However,  if  x is  very 
high,  the  correction  term  will  become  smaller  than  -S  and  the  optimal  control 
will  be  negative.  In  other  words,  if  inventory  level  is  coo  high,  the  factory 
can  save  money  by  disposing  a part  of  the  inventory  resulting  in  lower  holding 
costs . 

2)  If  the  demand  rate  S were  time-dependent,  it  would  have  changed  the 

solution  of  (24).  Having  computed  this  new  solution  in  place  of  (28),  we  can 

* 

once  again  obtain  the  optimal  control  as  u * Qx  + R/2. 

3)  Note  that  when  T -•  y - 0 and 

u — S - x.  (31) 

But  the  undiscounted  objective  function  value  (4)  in  this  case  becomes  (-<*>)  . 
Clearly,  any  other  policy  will  render  the  objective  function  value  to  be  -<*. 

In  a sense,  the  optimal  control  problem  becomes  ill-defined.  One  way  to  get  out 
of  this  difficulty  is  to  impose  a nonzero  discount  rate.  This  will  be  carried 
out  in  the  next  section. 

5.  SOLUTION  OF  THE  DISCOUNTED  INFINITE  HORIZON  CASE 

When  p > 0,  it  is  convenient  to  express  the  value  function  V(t,x) 
in  time-t  dollars,  i.e., 

W(t,x)  - V(t,x)ept.  (32) 

With  this  definition  of  W(t,x),  we  can  easily  convert  (14),  (15)  and  (16), 
respectively,  as  follows: 

0 - max[-(u2+x2)  - pW  + W + W (u-S)  + \ cj2W  ] , (33) 

U 1 X i XX 

« - V,/2 

0-^/4-  ,2-  SWx  4 i . 


(34) 
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To  solve  (34),  we  let 

W(t,x)  * Q(t)x2  + R(t)x  + M(t) 


and  with  a procedure  similar  to  that  used  in  deriving  (23)-(25), 
that  Q(t) , R(t),  and  M(t)  satisfy  the  following  system: 


Q * 1 + pQ  -Q 


R - pR  + 2SQ  - RQ  , 

M - pM  + RS  - R2/4  - a2q. 


Q(T)  - 0 
R(T)  * B 
M(T)  = 0 


It  can  be  shown  that  the  solution  for  Q(t)  is  given  by 


Q(t) 


^(■i-mjjT  . («i-m2)t 

e(mi-m2)T  _ m2  e(m1-m2)t 

“i 


> 


where, 


and  m, 


+ /o2^ 

2 


are  the  roots  of  1 + pQ  - Q"  ■ 0.  It  is  clear  that  as  T -•  <*>, 


Q - » 

since  < 0.  This  is  what  is  to  be  expected  since  m^ 

root. 

We  note  some  simple  Identities  that  m^  and  m^  satisfy 
used  later.  First  the  equation 

m2  - pm  ■ 1 


(35) 

we  can  show  that 

(36) 

(37) 

(38) 

(39) 

is  the  stable 

which  will  be 

(40) 
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holds  for  m being  either  or  m2.  Writing  (40)  for  m2  and  dividing  by 

2 

m2  gives 


Writing  (40)  for  and  dividing  both  sides  by  n^-p  gives 


(41) 


(42) 


Then,  since  m2  > 0,  and  the  product  of  and  m 2 is  -1  we  have 


ml  “ ’ ™ < °- 


(43) 


The  last  identity  needed  is 


m • 1+pm,  ■ 1 - — ^ 

1 1 n>2 


(44) 


which  can  be  obtained  by  writing  (40)  for  n^,  solving  for  m2  and  using 
(43). 

We  now  make  an  important  observation.  When  T * ®,  it  is  obvious  that 

W(t,x)  - W(x) 

Is  explicitly  Independent  of  t.  This  is  because  W is  expressed  in  time-t 
dollars.  This  means  Wfc  ■ 0,  which  reduces  (34)  to 


0 - W2/4  - x2  - SW  + £ <j2W  . 
* x 2 xx 


The  solution  of  (41)  takes  the  form 


(46) 


W(x)  - Qx  + Rx  + M 


(47) 
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where  Q,  R and  M are  constants  to  be  determined. 

conclude  from  (36)  that  Q,  R,  and  M satisfy 

It  is  not  difficult  to 

1 + pv/  - ^2  ■ 0 

(48) 

pR  + 2SQ  - RQ  - 0 

(49) 

pM  + RS  - R2/4  - <j2Q  - 0. 

(50) 

Of  the  two  roots  of  Q for  the  quadratic  equation  (48),  we  have  already 
shown,  by  a limiting  argument  that 


Q ” mr 

It  is  easy  to  show  using  (42)  that 


(51) 


2 

R - 2Sm^/ (m^-p)  - 2Sm1  * 2S(pmj+l)  (52) 

and 

M - [R2/4  - RS  + o2Q]/p  - [m2S2(m2-2)  + CMn^/p  (53) 

From  (33),-  we  can  obtain  the  optimal  production  rate 
* mlS 

u • V + "Is  + V ■ (1-  S^)s  ■ ^ x <*> 

Here  the  optimal  policy  is  to  produce  a positive  fraction  (1  - — ®)  times  the 

m2 

current  demand  plus  a correction  term  - (l/n^x  involving  the  inventory  level  x. 
Thus  it  is  optimal  to  produce  a high  level  when  inventory  is  low,  and  to  produce 
at  a low  level  when  inventory  is  high. 


Condition  for  Disposal 

From  (54),  it  is  possible  to  derive  necessary  and  sufficient  conditions 
for  disposal  or  negative  production  to  be  optimal.  We  have 
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■ k 

u < 


0 <*>  m^S  + m,x  < 0 


0 


(55) 


We  can  also  use  (44)  to.  write  the  condition  (55)  for  a disposal  to  be  optimal  as: 


u*<  0<*f>  - <p  + 3 


(56) 


Condition  (52)  may  be  compared  with  the  one  derived  for  the  deterministic  case 
in  [4].  Obviously,  a disposal  would  be  optimal  if  the  inventory  level  is  too 
high  compared  to  the  demand  rate.  This  makes  intuitive  sense. 

Relationship  with  the  Deterministic  Turnpike 

From  our  earlier  paper  [4]  we  know  that  x - -pS  is  the  turnpike  level 
for  the  inventory  level  and  u ■ S is  the  turnpike  production  rate  for  the  specie' 
problem  of  this  section  with  c * 0.  We  now  show  that  even  with  a + 0,  the 
optimal  production  rate  is  S when  the  observed  inventory  level  is  x ■ -pS. 

This  is  done  by  substituting  x ■ -pS  in  (54)  and  (40)  to  obtain 

* 2 

u • (m^  - pn^)S  ■ S . 

Summarizing  we  have 

u ■ S when  x ■ -pS  . (57) 

This  means  that  the  trajectory  for  the  optimal  Inventory  level  wo^ld  be  a dif- 
fusion process  about  x “ -pS  (see  Figure  1).  In  this  sense,  x “ -pS  can  be 
considered  to  be  the  turnpike  inventory  level  for  the  stochastic  production 
planning  problem  dealt  with  in  this  section. 
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Fig.  1.  A Sample  Path  for  Optimal  Inventory  Level 

The  sample  path  in  Figure  1 shows  a path  with  high  initial  inventory, 
which  increases  (due  to  sales  returns)  up  to  the  disposal  level.  Between  points 
a and  b,  production  is  negative,  which  means  we  dispose  of  inventory.  However, 
the  inventory  continues  to  increase  until  it  drops  back  at  point  (b).  Disposal 
activities  help  bring  back  the  inventory  to  lower  levels.  Further  on  the  path 
the  inventory  declines  randomly  to  the  turnpike  level  -pS  and  remains  in  that 
vicinity  with  high  probability. 

The  fact  that  the  turnpike  level  is  negative  is  because  x^  ■ 0 was 
assumed  in  (3).  When  x^  is  sufficiently  positive  the  turnpike  level,  x^  - pS, 
will  become  positive.  Details  are  omitted  here.  The  deterministic  case  is 
treated  in  [4] . 

6.  EXTENSIONS  AND  CONCLUDING  REMARKS 

We  have  analyzed  a simple  stochastic  production  planning  in  detail.  We 
have  deliberately  chosen  a simple  problem  to  introduce  the  stochastic  control 
theory  methodology,  which  may  not  be  generally  familiar.  The  methodology  is 
baaed  on  the  theory  of  stochastic  differential  equations  developed  by  Ito. 
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In  our  future  papers,  we  shall  analyze  more  complicated  stochastic 
production-inventory  problems.  These  will  include  problems  which  assume 
time-dependent  demand  or  a constant  diffusion  coefficient,  which  may  also 
be  a function  of  the  inventory  level. 
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